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We study the stable attractors of a class of continuous dynamical systems that may be idealized 
as networks of Boolean elements, with the goal of determining which Boolean attractors, if any, are 
good approximations of the attractors of generic continuous systems. We investigate the dynamics 
in simple rings and rings with one additional self- input. An analysis of switching characteristics 
and pulse propagation explains the relation between attractors of the continuous systems and their 
Boolean approximations. For simple rings, "reliable" Boolean attractors correspond to stable con- 
tinuous attractors. For networks with more complex logic, the qualitative features of continuous 
attractors are influenced by inherently non-Boolean characteristics of switching events. 
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I. INTRODUCTION 

Complex networks of interacting elements arise in 
many biological, chemical, sociological, and physical con- 
texts. An important example is the network of interac- 
tions among proteins and DNA in a cell. The binding of 
certain proteins to each other and to promoter regions of 
DNA can create a combinatorially complex logic of gene 
expression. It is tempting to think of transcriptional net- 
works and similar examples as effectively Boolean in na- 
ture. In such a picture, genes are turned "on" and "off" 
in the presence ofproteins produced when other genes are 
turned on or off |l| , and many studies of the fundamental 
principles governing networks of interacting genes have 
focused Boolean models. Recent work has highlighted 
distinctions in the attractor structures as network archi- 
tecture parameters are varied, different distributions of 
Boolean functions are incorporated, and /or different up- 
dating schemes are employed 0, H, 0, H, H, 0> Hi • 

The Boolean models are generally understood to be 
idealized representations of underlying continuous and 
perhaps stochastic processes, so it is important to un- 
derstand any artifacts introduced in Boolean approxima- 
tions. Here we investigate some continuous, deterministic 
systems motivated by models of transcriptional interac- 
tions and designed to be good candidates for a Boolean 
analysis. The goal is to elucidate the most important 
effects that may lead the continuous dynamics to dif- 
fer qualitatively from expectations based on the Boolean 
models. 

We investigate the temporal structure and stability of 
the attractors of continuous dynamical systems and those 
of the corresponding Boolean models. Two effects are 
found to be crucial for understanding the structure of the 
continuous attractors: first, when an ON-OFF symmetry 
of typical Boolean models is broken, the possibility of 
stable pulse propagation down a chain can be lost; and 
second, the memory of past inputs at a given node causes 
shifts in the temporal spacing between multiple pulses on 
a feedback loop. These effects make it difficult to put the 
attractors of the continuous network into correspondence 



with the attractors of their Boolean counterparts. 

We study rings of N elements (Fig. [1]) governed by 
delay differential equations. The delays are introduced 
to represent intermediate steps in the process mediating 
the interactions between elements. We employ a form 
developed originally as a mean-field description of the 
dynamics of transcription factor expression Q, though 
for present purposes it simply provides a generic model 
of elements with sigmoidal responses to their inputs: 



fj{Xi) = rjj 



Xj(t), 



1 + 



(1) 
(2) 



where rjj, b^, and d\ are constants and r is a time re- 
quired for the signal produced by Xj-i to reach Xj. All 
subscripts are taken modulo N. The parameters are cho- 
sen such that the output of fj switches sharply between 
low and high values as the input is smoothly varied. We 
assume that all Xj have the same decay rate (chosen to 
be unity) and all arguments of fj have the same time 
delay. We also consider the effects of adding a self-input 
to x\, also with delay r: 



xi(t) = fi(x {t - r),xi(t - t)) - xi(t) , (3) 
' 1 + djxg + d\x\ + d^{x xi) u / 2 ' 



fl(x ,Xi) = 77i 



l + blx^ + blaZ + bl^xoxi)"' 3 



(4) 



A Boolean idealization is obtained in the limit in which 
fj is a step function and the decay term —Xj(t) has an 
infinite coefficient. 

We find that continuous systems can exhibit stable os- 
cillations in cases where Boolean reasoning would suggest 
otherwise and that in some cases where Boolean reason- 
ing predicts a stable attractor, the corresponding contin- 
uous attractor does not have the expected structure. 
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II. BOOLEAN SYSTEMS 

We begin by summarizing the known behavior of 
Boolean systems where each xj is taken to be a Boolean 
variable and its dependence on its inputs is specified by 
a Boolean function Fj. A common choice is to update 
all the elements synchronously, setting each Xj at each 
discrete time step to the value Fj returned just after the 
previous step. Synchronously updated systems are easy 
to simulate, but are not generic. To avoid artifacts of 
synchronicity, Klemm and Bornholdt (Io| present a class 
of autonomous Boolean systems running in continuous 
time. Here the output of each Fj is fed through a filter 
that delays the signal by a fixed time (analogous to our 
r, which they set equal to 1) and cuts out short pulses. 
(See Appendix [2] for details.) These autonomous net- 
works have state cycles that correspond to the attractors 
of a synchronously updated Boolean network. In contrast 
to synchronous Boolean networks, however, autonomous 
networks permit the study of infinitesimal fluctuations in 
the timing of switching events. One can externally im- 
pose a delay in one switching event and see whether the 
sequence of time intervals between switching events is re- 
stored by the dynamics. If all possible small time delays 
evolve back to the same sequence of switching times, the 
state cycle of the autonomous system is stable. If, on 
the other hand, a subset of the switching times in the 
cycle remain delayed compared to the others for some 
perturbation, the state cycle is marginally stable. There 
are no unstable state cycles, because there is no way for 
an infinitesimal perturbation to get amplified. As a con- 
sequence, autonomous networks have an infinite set of 
marginally stable cycles. 

Klemm and Bornholdt coined the terms "reliable" and 
"unreliable" to denote attractors in the synchronous sys- 
tem that correspond to stable and marginal cycles, re- 
spectively, in the corresponding autonomous system [l(| • 
In accordance with their convention, we use the term "at- 
tractor" for any periodic state-cycle in a synchronously 
updated network. Unreliable attractors are not expected 
to be observed in real systems because errors in timing 
can accumulate and eventually cause a transition to a dif- 
ferent attractor. One of our goals is to determine whether 
the set of reliable attractors is in one-to-one correspon- 
dence with the set of attractors of a continuous system. 

The dynamics of simple Boolean rings have been well 




FIG. 1: A simple ring and a self-input ring. C indicates that 
the element copies its input; F indicates that the element 
is either a copier or an inverter on the simple ring, or that it 
performs one of the two-input logic functions on the self-input 
element. 



characterized [J, [T(| [ll[ ■ In a simple ring, each element 
either copies or inverts the state of its input. A ring with 
an even (odd) number of inverters is dynamically equiv- 
alent to a ring with zero (exactly one) inverters since a 
pair of inverters can be transformed to copiers by redefin- 
ing the meaning of ON and OFF for all elements between 
them. For a ring with no inverters, there are two fixed 
points; the states with all elements ON or all elements 
OFF. For a ring with one inverter there is no fixed point 
because at all times there must be at least one element 
whose value is not consistent with its input. We refer to 
this local inconsistency as a "kink" and we identify the 
kink as "positive" ("negative") when the element's in- 
put is ON (off). A single kink traveling around the ring 
forms a stable cycle with the kink changing its sign each 
time it passes the inverter. For synchronous updates, the 
separation between kinks cannot change, so every state 
lies on an attractor; there are no transients. For the 
autonomous system, any cycle with more than one kink 
is marginally stable because there is no restoring mech- 
anism for a perturbation in the time lag between two 
kinks. The multi-kink synchronous attractors are there- 
fore unreliable. 

For rings in which element 1 has a self-input in addition 
to its input from element 0, the attractor structure is 
nontrivial. Without loss of generality, we assume that 
the remaining elements are all copiers. There are ten 
possible choices of the Boolean function Fi at element 1 
for which both inputs are relevant. These comprise five 
pairs that are related by an ON-OFF symmetry. For two 
of these pairs, the only attractors are fixed points, so 
there are only three nontrivial cases; (I) F\ — NOR, (II) 
Fi = XOR, and (III) F 1 — (x and not xi). 

For the present discussion, we restrict attention to the 
case N = 4. In Case I, there is a single attractor that is 
(surprisingly) unreliable. In Cases II and III, the all-OFF 
state is a fixed point. Case II also has a reliable cycle 
containing all 15 other states. Case III has two cyclic 
attractors, both unreliable: a 4-state cycle consisting of 
the state 1000 and its cyclic permutations; and a 2-state 
cycle consisting of the states 1010 and 0101. 

Our first important observation is that the definition 
of reliability employed by Klemm and Bornholdt assumes 
the time delay for an element is the same regardless of 
whether an input is turning ON or off. The breaking of 
this symmetry (see below) leads to different propagation 
speeds for positive and negative kinks, which can destroy 
or stabilize some marginally stable cycles. For example, 
the unreliable 4-cycle in Case III is stabilized if positive 
kinks move faster than negative kinks. Though the du- 
ration of the ON pulse increases as it goes around the 
ring, it is cut back to r each time the pulse passes the 
self- input element. 
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III. ANALYSIS OF CONTINUOUS SYSTEMS 

We now turn to the analysis of continuous systems de- 
scribed by Eqs. CQ)-(l4|). We choose the Hill coefficient 
(or cooperativity) v = 2 corresponding to regulation per- 
formed by dimers @ . We fix 77, b, and d as follows in order 
to observe clearly identifiable off and ON states. For a 
single element that receives one input, we set 

77 = 1, b = 0.001, d = 0.1 for a copier; , . 

77 = 100, b = 0.1, d = for an inverter. ^ ' 

With these choices, there are three fixed points for sim- 
ple rings with only COPY elements: the all-OFF state 
x = 1.13; the all-ON state x = 88.9; and the unsta- 
ble switching value x = 10.0. For an invert element, 
a steady OFF input of 1.13 produces x = 88.7 (close to 
the ON value); an ON input of 88.9 produces x = 0.13 
(close to the OFF value); and x — 9.67 is an unstable 
fixed point. For the self- input element, the values listed 
in Table IIIII are chosen to represent the classes of func- 
tions whose Boolean idealizations would be Cases I, II, 
and III above. 

To study the propagation of positive and negative 
kinks, we derive the time it takes for a single kink to 
pass from one element to the next in a chain of single- 
input elements. The expression levels are denoted by 
xo{t),xi(t), X2(t), . . ., where xg(t) serves as an input and 
the other elements obey Eq. (Q]). For initial conditions, 
we assume that Xj (t) has the fixed value ctj for all t < 0, 
with ctj = fj[aj_x) for j = 1, 2, . . .. 

If Xq switches values, approaching a constant value (3q 
at long times, each of the Xj will eventually approach a 
new value (3j. For mathematical convenience, we define 
rescaled quantities Xj{t) = [xj(t) — 0j]/(oij — (3j) and 
fj(x) = [fj(x)—0j]/(aj—/3j) with the properties Xj (0) = 
1 and Xj(t) — ► for large t. We further define a specific 
time Sj associated with the switch from otj to (3j by the 
formula 



100 



dt Xj (t) . 



The formal solution of Eq. ([TJ) is 



,(<)=/ dwij(^-i(u-T))e u - 

J —OO 



(6) 



(7) 



for all j ^ 0. Substituting into Eq. © and subtracting 
Sj_i from both sides yields an expression for the time 



TABLE I: Parameter values for self-input elements. 
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61 


boi 


dl 


dl 


doi 
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100 


0.1 


0.1 














II 
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0.001 


0.001 


0.02 


0.1 


0.1 





III 


1 


0.001 





0.02 


0.1 










400 



410 



420 



430 



FIG. 2: Propagation of a single kink around a ring of four 
elements with one inverter and r = 2. The thick, thin, dashed, 
and grey lines represent element 1, 2, 3, and 0, respectively. 
Note that propagation is faster when the kink is positive and 
slower after the inverter converts it to a negative kink. 



delay between the switching of elements J — 1 and j: 

Sj - Sj.! = T + 1 + / dt [fj( X j ;(/!) - Xj^(t)], (8) 

Jo 

This delay is the sum of the explicit delay r, an intrinsic 
delay of unity associated with the unit coefficient of the 
—Xj term of Eq. (fTJ), and an additional term that depends 
on the details of the input function fj. 

For chains of identical copiers with the parameters 
specified above, it is straightforward to show that the 
additional delay is positive (negative) for negative (pos- 
itive) kinks and therefore that positive kinks will propa- 
gate faster, which implies that an ON pulse will expand 
in width and an off pulse will shrink and disappear. 
By changing the parameters 77, b, and d, it is possible to 
reverse this situation, but arranging for precisely equal 
propagation speeds requires fine tuning. 

In the numerical simulation of Fig. it is clear that 
the kink moves faster in its positive form than in its neg- 
ative form. (To bring out the asymmetry, this figure was 
made for r = 2, a relatively short time. Though the 
propagation time from one site to the next depends on r, 
the difference between propagation times for positive and 
negative kinks does not.) The asymmetry is also present 
in the continuous two-element rings in Ref. [L2| and in 
the repressilator simulation of Ref. [Hj]. The electronic 

14j, however, 



model with step function switching in Ref. 
does not break the symmetry for the case studied, in 
which the switching level is halfway between the ON and 
OFF voltages. 

Until now, we have neglected the fact that the propaga- 
tion speed of a kink through a given element is influenced 
by kinks that have previously passed through. Consider 
now an ON pulse consisting of a positive kink followed by 
a negative kink. From Eq. ([7|), we see that Xj(t) has an 
exponentially decaying memory of the events in Xj-\{t). 
Because fj(x) is monotonic, the memory of low values 
of Xj—i(t) before the pulse in Xj-\(t) will speed up the 
effect of the trailing edge. The interaction shortens the 
pulse duration at Xj(t), and monotonicity of fj ensures 
that the effect is stronger for shorter pulses. Similar rea- 
soning shows that an off pulse will also be shorter than 
the time between kinks one would predict from Eq. 
alone. 
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In a chain of copiers, different propagation speeds of 
positive and negative kinks will lengthen (shorten) a trav- 
eling ON (off) pulse. In a chain that contains inverters 
arranged such that a single pulse spends equal amounts 
of time in its ON and OFF configurations, the asymme- 
try between positive and negative kinks alone may not 
change the average pulse duration, but the pulse would 
still be shortened by the memory effect. 

Because the strength of the memory effect increases 
as two kinks approach each other, a pulse or other se- 
quence of kinks cannot propagate stably on a chain of 
single-input elements — only a single kink can have a 
stable shape as it advances. For simple rings, then, sta- 
ble attractors must have only zero or one kink. These 
two possibilities correspond precisely to the fixed points 
and single-kink cycles that are the reliable attractors of 
the corresponding Boolean systems. 

For rings with a self- input (Fig. [1]), the situation is 
more complicated. First, note that the analysis of pulse 
propagation gives a quantitative measure of the asym- 
metry discussed above. The asymmetry enables some 
attractors classified as unreliable by the (symmetric) 
Boolean analysis to be stable in the continuous sys- 
tem. Second, memory effects associated with multiple 
inputs to a single element can lead to repulsion between 
pulses and stabilization of new attractors related to syn- 
chronous Boolean ones but with shifts in the timing be- 
tween pulses. 



IV. NUMERICAL RESULTS 

We studied the three cases numerically using an in- 
tegration scheme that takes advantage of the structure 
of Eq. ([7]) as described in Appendix [Bj In Case I, most 
initial conditions lead first to a long transient that corre- 
sponds to the unreliable attractor observed in the syn- 
chronous Boolean network. The transient eventually 
gives way to a stable attractor of Fig.[3la), which has two 
pulses separated in time by approximately (5/2)(r + 1), 
an example of the stabilization of an intermediate inter- 
pulse interval where the synchronous Boolean attractor 
has pulses separated in time by alternating intervals of 
2 and 3 time steps. We note in passing that the path 
by which the continuous system arrives at the attrac- 
tor is rather robust: for a wide range of initial condi- 
tions, the system goes to the Boolean-like transient in a 
short time and then gradually shifts to the attractor. In 
some cases, however, we observe the attractor shown in 
Fig. [3](b) , which has three pulses with a time separation 
of (5/3) (r + 1) between each pair. This attractor cor- 
responds to a marginally stable cycle of the autonomous 
Boolean network, but the timing is incompatible with the 
synchronous updating scheme. 

We note that the appearance of attractor periods hav- 
ing 5(r+l) as an integer multiple can be understood an- 
alytically. Kaufman and Drossel have analyzed the pos- 
sible attractor periods in synchronous Boolean networks 
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FIG. 3: Time series for Cases I (a and b), II (c), and III (d 
and e), with r = 10. The thick, thin, dashed, and grey lines 
represent element 1, 2, 3, and 0, respectively. Case I shows (a) 
non-synchronous timing stabilization due to memory effects 
(with time axis labels shifted by 10 6 to account for a long 
transient) and (b) a stable attractor that has no analogue 
in the corresponding synchronous Boolean network. Case II 
shows (c) an inherently non-Boolean attractor. Case III shows 
(d) single pulse and (e) double pulse attractors, both of which 
are unreliable in a Boolean network. 



in rings with a single cross-link [151 ] . A straightforward 
extension of their method to the autonomous networks 
of our ring with a NOR self-input (Case I) reveals that all 
cycles in the autonomous model must have a period of 
the form 5 /to, where m is an integer and the time unit is 
the time required for a kink to advance through one ele- 
ment. The corresponding period in the continuous model 
is therefore 5(r + l)/m. 

In Case II, two attractors are observed: the fixed point 
and an oscillatory attractor that appears to be aperi- 
odic. The latter behavior is sensitive to intermediate 
variable values and is thus inherently non-Boolean. In 
Case III, we observe one fixed point and two limit cycles, 
corresponding to each of the three Boolean attractors, 
with no long transients. The limit cycles are unreliable 
in the Boolean case but stable in the continuous case 
- examples of stabilization due to asymmetry in kink 
propagation speeds and a pulse-shortening self-input ele- 
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merit. This is the effect alluded to above at the end of the 
Boolean systems section. Though the pulse has broad- 
ened in traveling around the ring, a new trailing edge is 
generated by the self- input, which occurs one delay time 
after the arrival of the leading edge. 

The difference in pulse height in the two limit cycles is 
due to the dependence of fi(xo,xi) on small variations 
in x\ when x$ is nominally ON and x\ is nominally off. 
The variations in the OFF-state of X\ are caused by the 
delayed self-input that suppresses the OFF-state during 
a time of approximately r following each pulse. In the 
limit cycle with two pulses, the suppression is still active 
when the next pulse starts. This is not the case for the 
single-pulse attractor and the slightly higher OFF-state 
leads to a suppression of the pulse height. 

The above observations remain valid for all sufficiently 
large values of r. For small r, the systems studied have 
only fixed point attractors. As r is increased, the oscil- 
latory attractors are born by means of saddle-node bi- 
furcations of cycles [l6| in Cases II and III. In Case I, a 
subcritical bifurcation to a state cycle with period near 
3 x (5/2)(r + 1) is followed by a subcritical, symmetry- 
restoring bifurcation with period near (5/2)(r + 1). The 
cycle with period near 3 x (5/2)(r + 1) has no direct 
correspondence to a cycle in the autonomous Boolean 
network. 



dilations in yeast, where there is evidence for a tran- 
scriptional oscillator and recent proposals for the genes 
involved suggest a fundamental ring of four elements with 
multiple feed- forward and feedback links [l9|. We have 
seen, for example, that a pulse of activity may propa- 
gate stably in such networks even where Boolean reason- 
ing suggests otherwise. In systems where distinct ele- 
ments have significantly different time delays, we expect 
additional deviations from the dynamics in synchronous 
or autonomous Boolean networks. Future studies along 
these lines should elucidate the behavior of larger rings 
and more complex network structures. 
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APPENDIX A: AUTONOMOUS DYNAMICS 



V. DISCUSSION 

Our study of simple and single-self-input rings has elu- 
cidated some important non-Boolean features of contin- 
uous systems: (i) the asymmetry in the reaction time of 
an element when an input switches ON or off; (ii) devia- 
tions from nominal ON and OFF values; and (iii) memory 
effects due to the exponential decay of variables to their 
steady state values for fixed inputs. These features are 
crucial for the stabilization and destabilization of oscil- 
lations in the systems we have investigated. In simple 
rings, they lead to a set of stable attractors that coin- 
cides precisely with the reliable attractors identified by 
Klemm and Bornholdt [1 0f | . When more complex logic is 
introduced, as illustrated in here by adding a self-input 
to one element in a ring, the stable continuous attractors 
do not correspond to the reliable attractors. 

These observations are important for generalizing the 
well-developed theory of large random Boolean networks 
to generic systems. We would like to know, for example, 
whether large, complex networks exhibit a well-defined 
dynamical phase transition similar to the "order-chaos" 
transition in ensembles of Boolean networks 0, [l?], OH • 
An important feature of the Boolean network transition 
is the rapid scaling of the number of attractors with sys- 
tem size in the disordered regime, which suggests that 
we should try to understand the set of attractors of large 
and complex continuous systems. 

The non-Boolean effects discussed in this paper may 
also be directly relevant for understanding cell cycle os- 



For completeness, we describe the implementation of 
autonomous Boolean systems introduced by Klemm and 
Bornholdt [10j. The important features are that switch- 
ing times are determined by local time delays and that 
repeated switching on times scales much shorter than the 
delay time is suppressed. There is no external clock and 
no stochastic rule for determining update times. 

As in Ref. [l(|, we assume a time delay of 1 unit be- 
tween the switching of a given node and the switching it 
induces in nodes directly linked to it. At each node, a 
low-pass filter is assumed to suppress spikes of duration 
much shorter than 1. We let e be the minimum dura- 
tion of an output pulse that passes the filter (and require 
< e C 1). If the inputs to a given node switch at times 
that would lead to a pulse shorter than e, the output 
from that node is assumed to stay constant during that 
time. 

For notational convenience, we write Fj(t), suppress- 
ing its direct dependence on its inputs. We also let the 
Boolean values OFF and ON, respectively, correspond to 
the real values and 1 . Then, Xj (t) is given by 



x j (t) = Q 



1 / 



(Al) 



for 



where O(x) is the Heaviside step function; 9(x) 
x < and 1 for x > 0. 

When the timing of some switching events are per- 
turbed infinitesimally, Fj may switch twice in rapid suc- 
cession, which would generate a positive or negative pulse 
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of infinitesimal duration. The filter of Eq. (|A1|) sup- 
presses such spikes, ensuring that the number of switch- 
ing events remains constant (for cycles in which all spikes 
are longer than e in duration) and therefore that the sta- 
bility of a state cycle is well defined with respect to in- 
finitesimal timing perturbations. 

APPENDIX B: NUMERICAL INTEGRATION 
METHOD 

Numerical integration of the time-delay equations |T])- 
(|4]) was accomplished using a fourth-order scheme based 
on the solution shown in Eq. (J7J). To evolve the system 
through a time step h, we define values of x at the time 
points t — nh for all integer n and define m = r/h, 
where r is the delay time and h is chosen such that m 
is an integer. To advance from time step n — 1 to n we 
use the following formula (suppressing the element index 
subscript j for notations! simplicity): 

%n — 2/l I ~x fn—m 77^ fn—m—1 "h 77^ fn—m—2 ) 
\6 3 6 / 

+ e~ 2h x n - 2 , (Bl) 



where /„ is the function defined by Eq. ((2|) [or eval- 
uated at time t = nh. The integrator is initialized with a 
desired value of xq , where it is assumed that x n = xq for 
all n < 0. Integrations were carried out using h = 0.1. 
Decreasing h to 0.01 had no noticeable effect on the re- 
sults. 

Attractors were found by running from many (of order 
50) different initial conditions. In some cases, such as the 
attractor shown in Fig.[3jb), it was necessary to arrange 
initial conditions in which some nodes were artificially 
held at constant values and released at different times. 
The bifurcation structures as a function of the time-delay 
parameter r were determined by performing integrations 
in which h was increased or decreased very slowly and 
observing transitions in the oscillation patterns of all x's. 
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